Proxying for Unobservable Variables with Internet Documentfrequency

The internet contains billions of documents. We study if there is useful information in the frequency with which different topics are written about. Based on the premise that the occurrence of an event increases its textual frequency, we assess whether internet document-frequency can capture cross-sectional variation in the occurrence-frequency of social phenomena. We characterize the conditions under which such proxying is likely. We successfully proxy for a number of demographic variables at the US city and state levels. We obtain document-frequencybased measures of corruption at the country and state level and replicate the results of previous research studying its covariates. Finally, we illustrate the usefulness of the approach by creating the first index of corruption in US cities. JEL: H00, J11, C81, B40, D73